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Abstract 

We discuss and analyze a list-message-passing decoder with verification for low-density parity-check (LDPC) codes on the 
q-ary symmetric channel (q-SC). Rather than passing messages consisting of symbol probabilities, this decoder passes lists of 
possible symbols and marks some messages as verified. The density evolution (DE) equations for this decoder are derived and 
used to compute decoding thresholds. If the maximum list size is unbounded, then we find that any capacity-achieving LDPC 
^v^j ' code for the binary erasure channel can be used to achieve capacity on the q-SC for large q. The decoding thresholds are also 

computed via DE for the case where each list is truncated to satisfy a maximum list size constraint. Simulation results are also 
presented to confirm the DE results. During the simulations, we observed differences between two verification-based decoding 
algorithms, introduced by Luby and Mitzenmacher, that were implicitly assumed to be identical. In this paper, we provide an 
analysis of the node-based algorithms from that paper and verify that it matches simulation results. 

The probability of false verification (FV) is also considered and techniques are discussed to mitigate the FV. Optimization 
of the degree distribution is also used to improve the threshold for a fixed maximum list size. Finally, the proposed algorithm is 
compared with a variety of other algorithms using both density evolution thresholds and simulation results. 



I. Introduction 

O . Low-density parity-check (LDPC) codes are linear codes that were introduced by Gallager in 1962 (2) and re-discovered 
by MacKay in 1999 [3'j. The ensemble of LDPC codes that we consider (e.g. see J4) and 0) is defined by the edge degree 
£f~) . distribution (d.d.) functions X(x) = Ylk>2 ^fe 3 - an< ^ p( x ) = J2k>2 Pk% k ~ l ■ The standard encoding and decoding algorithms 
are based on the bit-level operations. However, when applied to the transmission of data packets, it is natural to perform the 
^) , encoding and decoding algorithm at the packet level rather than the bit level. For example, if we are going to transmit 32 bits 
as a packet, then we can use error-correcting codes over the, rather large, alphabet with 2 32 elements. 
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00 , where x (resp. y) is the transmitted (resp. received) symbol and x, y £ GF(q). The capacity of the q-SC is 1 + (1— p) log q (l — 
p) + plog q p — plog q (q — 1) which is approximately equal to 1 — p for large q. This implies the capacity of the g-SC with 
large q is approximately equal to the capacity of the BEC with erasure probability p. Moreover, the behavior of the q-SC with 
large q is similar to the BEC in the sense that: i) incorrectly received symbols from the g-SC provides almost no information 
about the transmitted symbol and ii) error detection (e.g., a CRC) can be added to each symbol with negligible overhead [6]. 

Binary LDPC codes for the g-SC with moderate q are optimized based on EXIT charts in [7| and |8|. It is known that the 
complexity of the standard belief-propagation algorithm for g-ary LDPC codes scales like q log 2 q. Even for moderate sizes of 
g, such as g = 256, this renders such algorithms ineffective in practice. However, when q is large, an interesting effect can be 
used to facilitate decoding: if a symbol is received in error, then it is essentially a randomly chosen element of the alphabet, 
and the parity-check equations involving this symbol is very unlikely to be valid. 

In Q~), Luby and Mitzenmacher developed an elegant algorithm for decoding LDPC codes on the g-SC for large g. However, 
their paper did not present simulation results and left capacity achieving ensembles as an interesting open problem. Metzner 
presented similar ideas earlier in [9] and [10|, but the focus and analysis is quite different. Davey and MacKay develop and 
analyze a symbol-level message-passing decoder over small finite fields [11 J. A number of approaches to the g-SC (for large 
g) based on interleaved Reed-Solomon codes are also possible [6| [12|. In [13], Shokrollahi and Wang discuss two ways of 
approaching capacity. The first uses a two-stage approach where the first stage uses a Tornado code and verification decoding. 
The second is, in fact, equivalent to one of the decoders we discuss in this paperQ When we discovered this, the authors were 
kind enough to send us an extended abstract [14| which contains more details. Still, the authors did not consider the theoretical 
performance with a maximum list size constraint, the actual performance of the decoder via simulation, or false verification 
(FV) due to cycles in the decoding graph. In this paper, we describe the algorithm in detail and consider those details. 



This research was supported in part by the National Science Foundation under Grant No. 0747470. 

'The description of the second method 1 1 3 1 is very brief and we believe its capacity-achieving nature deserves further attention. 



2 



Inspired by [Q], we develop in this paper list-message-passing (LMP) decoding with verification for LDPC codes on the 
q-SC. Instead of passing a single value between bit and check nodes, we pass a list of candidates to improve the decoding 
threshold. This modification also increases the probability of FV. So, we analyze the causes of FV and discuss techniques to 
mitigate FV. It is worth noting that the LMP decoder we consider is somewhat different than the list extension suggested in 
(T). Their approach uses a peeling-style decoder based on verification rather than erasures. In §T\, the algorithms are proposed 
in a node-based (NB) style but analyzed using message-based (MB) decoders. It is implicitly assumed that the two approaches 
are equivalent. In fact, this is not always true. In this paper, we consider the differences between NB and MB decoders and 
derive an asymptotic analysis for NB decoders. 

The paper is organized as follows. In Section II, we describe the LMP algorithm for bounded and unbounded list size and 
use density evolution (DE) [15| to analyze its performance. The difference between NB and MB decoders for the first (LM1) 
and second algorithm (LM2) in HI is discussed and the NB decoder analysis is derived in Section III and V, respectively. 
The problem of FV is considered in Section V. In Section VI, we use differential evolution to optimize code ensembles. We 
describe the simulation of these codes and compare the results with the theoretical thresholds. We also compare our results 
with previously published results in this area [1] and JT3). In Section VII, simulation results are shown. Applications of the 
LMP algorithm are discussed and conclusions are given in Section VIII. 

II. Description and Analysis 

A. Description of the Decoding Algorithm 

The LMP decoder we discuss is designed mainly for the q-SC and is based on local decoding operations applied to lists 
of possible codeword symbols. The messages passed in the graph have three types: verified (V), unverified (U) and erasure 
(E). Every V-message has a symbol value associated with it. Every U-message has a list of symbol values associated with it. 
Following JT), we will mark messages as verified when they are very likely to be correct. In particular, we will find that the 
probability of FV approaches zero as q goes to infinity. 

The LMP decoder works by passing list-messages around the decoding graph. Instead of passing a single code symbol (e.g., 
Gallager A/B algorithm |2)) or a probability distribution over all possible code symbols (e.g., [11]), we pass a list of values 
that are more likely to be correct than the other messages. At a variable node, the output list contains all symbols which could 
satisfy the check constraint for the given input lists. At the check node, the output message will be verified if and only if all the 
incoming messages are verified. At a node of degree d, the associativity and commutativity of the node-processing operation 
allow it to be decomposed into (d— 1) binar>0 operations (e.g., a+b+c+d= (a+b)+(c+d)). In such a scheme, the computational 
complexity of each binary-operation is proportional to s 2 at the check node and s In s at the variable nod^H, where s is the list 
size of the input list. The list size grows rapidly as the number of iterations increases. In order to make the algorithm practical, 
we have to truncate the list to keep the list size within some maximum value, denoted S max . In the algorithm analysis, we 
also find that, after the number of iterations exceeds half the girth of the decoding graph, the probability of FV increases very 
rapidly. We analyze the reasons of FV and classify the FV's into two types. We find that the codes described in (T) and lfl"3l 
both suffer from type-II FV. In Section V, we analyze these FV's and propose a scheme to reduce the probability of FV. 

The message-passing decoding algorithm using list messages (or LMP) applies the following simple rules to calculate the 
output messages for a check node: 

• If all the input messages are verified, then the output becomes verified with the value which makes all the incoming 
messages sum to zero. 

• If any input message is an erasure, then the output message becomes an erasure. 

• If there is no erasure on the input lists, then the output list contains all symbols which could satisfy the check constraint 
for the given input lists. 

• If the output list size is larger than S max , then the output message is an erasure. 
It applies the following rules to calculate the output messages of a variable node: 

• If all the input messages are erasures or there are multiple verified messages which disagree, then output message is the 
channel received value. 

• If any of the input messages is verified (and there is no disagreement) or a symbol appears more than once, then the 
output message becomes verified with the same value as the verified input message or the symbol which appears more 
than once. 

• If there is no verified message on the input lists and no symbol appears more than once, then the output list is the union 
of all input lists. 

• If the output message has list size larger than S max , then the output message is the received value from the channel. 

2 Here we use "binary" to emphasize that there are two inputs although the operation is over GF{q). 

3 The binary-operation at the variable node can be done by s binary searches of length s and the complexity of a binary search of length s is O(lns) 
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B. DE for Unbounded List Size Decoding Algorithm 

To apply DE to the LMP decoder with unbounded list sizes, denoted LMP-oo (i.e., S max = oo), we consider three quantities 
which evolve with the iteration number i. Let Xi be the probability that the correct message symbol is not on the list passed 
from a variable node to a check node. Let yi be the probability that the message passed from a variable node to a check node 
is not verified. Let Zi be the average list size passed from a variable node to a check node. The same variables are "marked" 
(xi, yi,Si) to represent the same values for messages passed from the check nodes to the variable nodes (i.e., the half-iteration 
value). We also assume all the messages are independent, that is, we assume the code length is infinite and there are no cycles 
in the bipartite graph. 

First, we consider the probability, a;,, that the correct message symbol is not on the list. For any degree-c? check node, the 
correct message symbol will only be on the edge output list if all of the other d — 1 input lists contain their corresponding 
correct symbols. This implies that Xi — 1 — p(l — a;,). For any degree-c? variable node, the correct message symbol will not be 
on the edge output list only if it is on none of the other d — 1 edge input lists. This implies that Xi+\ = pX(Si). This behavior 
is very similar to erasure decoding of LDPC codes on the BEC and gives the identical update equation 

x i+ i = pX (1 - p(l - Xi)) (1) 

where p is the q-SC error probability. Next, we consider the probability, j/j, that the message is not verified. For any degree-c? 
check node, an edge output message is verified only if all of the other d — 1 edge input messages are verified. For any degree-c? 
variable node, an edge output message is verified if any symbol on the other d — 1 edge input lists is verified or occurs twice 
which implies jji = 1 — p(l — y,). The event that the output message is not verified can be broken into the union of two 
disjoint events: (i) the correct symbol is not on any of the input lists, and (ii) the symbol from the channel is incorrect and 
the correct symbol is on exactly one of the input lists and not verified. For a degree-c? variable node, this implies that 

Pr(not verified) = (x^ + p(d - 1) (y f - Xi) (x l ) d ~ 2 . (2) 
Summing over the d.d. gives the update equation 

yi+i =A (1 - p(l - Xl )) +p( P (l - Xi) - P (l - A' (1 - p(l - Xl )) . (3) 

It is important to note that (Q~|) and (O were published first in lfl3l Thm. 2] (by mapping xi = pi and yi = pi + qi), but were 
derived independently by us. 

Finally, we consider the average list size z^. For any degree-c? check node, the output list size is equafl to the product of 
the sizes of the other d—1 input lists. Since the mean of the product of i.i.d. random variables is equal to the product of the 
means, this implies that z% = p(zi). For any degree-c? variable node, the output list size is equal to on^| plus the sum of the 
sizes of the other d—1 input lists if the output is not verified and one otherwise. Again, the mean of the sum of d — 1 i.i.d. 
random variables is simply d—1 times the mean of the distribution, so the average output list size is given by 

1 + ((ii) d_1 + P(d - 1) (m - ii) (xif- 2 ) (d - l)Si. 

This gives the update equation 

z l+ i =l + [x l X' {xi)+p (jji-Xi) (A' (x l ) + x l X" {ii))] p(zi). 
For the LMP decoding algorithm, the threshold of an ensemble (X(x),p(x)) is defined to be 



p* = sup pe(0,i] 



pX(l-p(l-x)) <iVie(0,l] 

Next, we show that some codes can achieve channel capacity using this decoding algorithm. 

Theorem 2.1: Let p* be the threshold of the d.d. pair (X(x),p(x)) and assume that the channel error rate p is less than p*. 
In this case, the probability yi that a message is not verified in the i-th decoding iteration satisfies lim^oo j/j — > 0. Moreover, 
for any e > 0, there exists a q < oo such that LMP decoding of a long random (A, p) LDPC code, on a q-SC with error 
probability p, results in a symbol error rate of less than e. 

Proof: See Appendix A. ■ 

Remark 2.2: Note that the convergence condition, p*X(l — p(l — x)) < x for x £ (0, 1], is identical to the BEC case but 
that x has a different meaning. In the DE equation for the q-SC, x is the probability that correct value is not on the list. In the 
DE equation for the BEC, x is the probability that the message is an erasure. This tells us any capacity-achieving ensemble 
for the BEC is capacity-achieving for the q-SC with LMP-oo algorithm and large q. This also gives some intuition about the 
behavior of the q-SC for large q. For example, when q is large, incorrectly received value behaves like an erasure [6]. 

4 It is actually upper bounded because we ignore the possibility of collisions between incorrect entries, but the probability of this occurring is negligible as 
q goes to infinity. 

5 A single symbol is always received from the channel. 
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Corollary 2.3: The code with d.d. pair \{x) — x and p(x) = (1 — e)x + ex 2 has a threshold of 1 — and a rate of 
r > gTj+e)' Therefore, it achieves a rate of 0(6) for a channel error rate of p = 1 — 6. 

Proof: Follows from ^1 — j^j A (1 — p(l — x)) < x for x € (0, 1] and Theorem 12. II ■ 
Remark 2.4: We believe that Corollary 12.31 provides the first linear-time decodable construction of rate &(6) for a random- 
error model with error probability 1 — 6. A discussion of linear-time encodable/decodable codes, for both random and adversarial 
errors, can be found in fl6l . The complexity also depends on the required list size which may be quite large (though independent 
of block length). Unfortunately, we do not have explicit bounds on the required alphabet size or list size for this construction. 

In practice, we cannot implement a list decoder with unbounded list size. Therefore, we also evaluate the LMP decoder 
under a bounded list size assumption. 



C. DE for the Decoding Algorithm with Bounded List Size 

First, we list some definitions and notation for the DE analysis with bounded list size decoding algorithm. Note that in 
the bounded list size list decoding algorithm, each list may contain at most S max messages. For convenience, we classify the 
messages into four types: 

(V) Verified: message is verified and has list size 1. 

(E) Erasure: message is an erasure and has list size 0. 

(L) Correct on list: message is not verified or erased and the correct message is on the list. 

(N) Correct not on list: message is not verified or erased, and the correct message is not on the list. 
For the first two message types, we only need to track the fraction, Vi and Ei, of message types in the i-th iteration. For the 
third and the fourth types of messages, we also need to track the list sizes. Therefore, we track the characteristic function of 
the list size for these messages, given by Li(x) and Ni(x). The coefficient of x^ represents the probability that the message 
has list size j. Specifically, L{(x) is defined by 

Smax 

3 = 1 

where l^ j is the probability that, in the i-th decoding iteration, the correct message is on the list and the message list has size 
j. The function Ni(x) is defined similarly. This implies that is the probability that the list contains the correct message 

and that it is not verified. For the same reason, iVj(l) gives the probability that the list does not contain the correct message and 
that it is not verified. The same variables are "marked" (V, E, L, N and P) to represent the same values for messages passed 
from the check nodes to the variable nodes (i.e., the half-iteration value). For compactness, we denote the overall density as 
Pi = \Yi,Ek,Li(x),Ni(x)]. 

Using these definitions, we find that DE can be computed efficiently by using arithmetic of polynomials. For the convenience 
of analysis and implementation, we use a sequence of binary-operations plus a separate truncation operator to represent a 
multiple-input multiple-output operation. We use EH to denote the check-node operator and ® to denote the variable-node 
operator. Using this, the DE for the variable-node binary-operation P^ = ® P^ is given by 

03) = y(i) + v(2)_v(i)vP) + i(i)(i)L(s)(i) (4 ) 

E<® =E<»EW (5) 

L^ix) =LW(x) (e^+M 2 \x)) +L {2 \x) (e^+N {1) (x)) (6) 

N^\x) ^N^\x)E^+N {2 \x)E^+N [ - 1 \x)N {2) {x). (7) 

Note that Eq. to Eq. (0 do not yet consider the list size truncation and the channel value. For the binary check-node 
operation P^ = EE P (2) , the DE is given by 

yO) =v (i) v m (8) 

eW=eW+E<®-EWeW (9) 
L&(z) = \v^L^(z)+V^L^(z)+L^(x)L^(y) 

N<- 3 \z) = \ i N {1 \x)N {2 Hy) + N^(x) (v (2) y+L^(y) 

N^(x)(v^y+L^(y))} . (11) 

where the subscript x^y —> z^ means the replacement of variables. Finally, the truncation of lists to size S max is handled by 
truncation operators which map densities to densities. We use T and T' to denote the truncation operation at the check and 
variable nodes. Specifically, we truncate terms with degree higher than S max in the polynomials L(x) and N(x). At check 
nodes, the truncated probability mass is moved to E. 



xiy k — >zi k 
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TABLE I 

Brief Description of Message-Passing Algorithms for g-SC 



Alg. 


Description 


LMP-5 


LMP as described in Section III-AI with Smax =S 


LM1-MB 


MP decoder that passes (value, UIV). [1 III.B] 

At VN's, output is V if any input is V or message matches 

channel value, otherwise pass channel value. 
At CN's, output is V if all inputs are V. 


LM1-NB 


Peeling decoder with VN state (value, UIV). [J, III.B] 

At CN's, if all neighbors sum to 0, then all neighbors get V . 

At CN's, if all neighbors but one are V, then last gets V. 


LM2-MB 


The same as LM1-MB with one additional rule. (1 IV. A]. 
At VN's, if two input messages match, then output V. 


LM2-NB 


The same as LMI-NB with one additional rale. |T IVA]. 
At VN's, if two neighbor values same, then VN gets V. 


SW1 


Identical to LM2-MB 


SW2 


Identical to LMP-oo. |T3 Thm. 2] 



At variable nodes, lists longer than S max are replaced by the channel value. To analyze this, we separate L(x) into two terms: 
A{x) with degree less than S max and x Smam B(x) with degree at least S max . Likewise, we separate N(x) into C(x) and 
x Smal D(x). The inclusion of the channel symbol and the truncation are combined into a single operation 

pM=W V,E,A(x)+x s ^B(x),C(x)+x s ^D(x) ) 

defined by 

V (1) = V+{l-p)(A{l) +B(lj) (12) 



E {1) =0 (13) 
L {1) {x) = (l-p)x(jt+C(x)+D(lfj+pxA(x) (14) 

N w (x) =px (E+B(l)+C(x)+D(lf) . (15) 

Note that in Eq. l fT2l . the term (1 — p) (^A(l) + B(l)j is due to the fact that messages are compared for possible verification 
before truncation. 

The overall DE recursion is easily written in terms of the forward (bit to check) density P; and the backward (check to 
bit) density Pi. The initial density is Po = [0,0, (1 — p)x,px], where p is the error probability of the q-SC channel, and the 
recursion is given by 

k=2 

d v 

p l+ i =Y. Xk 

k=2 



r (pf fe - 1 ) . (i7) 



Note that the DE recursion is not one-dimensional. This makes it difficult to optimize the ensemble analytically. It remains an 
open problem to find the closed-form expression of the threshold in terms of the maximum list size, d.d. pairs and q. In section 
VI, we will fix the maximum variable and check degrees, code rate, q and maximum list size and optimize the threshold over 
the d.d. pairs by using a numerical approach. 



III. Differential Equation Analysis of LMI-NB 

A. Motivation 

We refer to the first and second algorithm in JT| as LM1 and LM2, respectively. Each algorithm can be viewed either as 
message-based (MB) or node-based (NB). The first and second algorithm in [13] and [14| are referred to as SW1 and SW2. 
These algorithms are summarized in Table U Note that if there is no verification occurring, the VN sends the ("channel value", 
U) and the CN sends the ("expected correct value",U). The algorithms SW1, SW2 and LMP are all MB algorithms, but can 
be modified to be NB algorithms. 

In m, the algorithms are proposed in the node-based (NB) style Q] Section III-A and IV] but analyzed in the message-based 
(MB) style [1, Section III-B and IV]. It is easy to see that the LMI-NB and LM1-MB are identical, but the NB and MB 
algorithms for LM2 are different. In this section, we will show the differences between the NB decoder and MB decoder and 
derive the correct analysis for LMI-NB. 

First, we show the equivalence between LM1-MB and LMI-NB. 
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Theorem 3.1: Any verification that occurs in LM1-NB also occurs in LM1-MB and vice versa. Therefore, LM1-NB and 
LM1-MB are equivalent. 

Proof: See Appendix B. ■ 
Remark 3.2: The theorem shows the equivalence between LM1-NB and LM1-MB. This also implies the error patterns or 
stopping sets of LM1-NB and LM1-MB are the same. 

In the NB decoder, the verification status is associated with the node. Once a node is verified, all the outgoing messages 
are verified. In the MB decoder, the status is associated with the edge/message and the outgoing messages may have different 
verification status. NB algorithms cannot, in general, be analyzed using DE because the independence assumption between 
messages does not hold. Therefore, we develop peeling-style decoders which are equivalent to LM1-NB and LM2-NB and use 
differential equations to analyze them. 

Following [4], we analyze the peeling-style decoder using differential equations to track the average number of edges 
(grouped into types) in the graph as decoding progresses. From the results from ifTTll and flU, we know that the actual number 
of edges (of any type), in any particular decoding realization is tightly concentrated around the average over the lifetime of 
the random process. In a peeling-style decoder for GF(q), a variable node and its edges are removed after verification. The 
check node keeps track of the new parity constraint (i.e., the value to which the attached variables must sum) by subtracting 
values associated with the removed edges. 

B. Analysis of Peeling-Style Decoding 

First, we introduce some notation and definitions for the analysis. A variable node (VN) whose channel value is correctly 
received is called a correct variable node (CVN), otherwise it is called an incorrect variable node (IVN). A check node (CN) 
with i edges connected to the CVN's and j edges connected to the IVN's will be said to have C-degree i and I-degree j, or 
type 

We also define some quantities as follows: 

• t: decoding time or the fraction of VNs removed from graph 

• Li(t): the number of edges connected to CVN's with degree i at time t. 

• Rj{t): the number of edges connected to IVN's with degree j at time t. 

• Nij(t): the number of edges connected to CN's with C-degree i and I-degree j. 

• Ei(t): the remaining number of edges connected to CVN's at time t. 

• E r (t): the remaining number of edges connected to IVN's at time t. 

• a(t): the average degree of CVN's, 

a{t) = y £ d L i {t)i/E l {t) 

• b(t): the average degree of IVN's, 

i>0 

• E: number of edges in the original graph, 

E = £7,(0) + E r (0) 
Counting edges in three ways gives the following equations: 

£ Li(t) + ft® = Ek (t) + E r (t) = X)E N V (*) ■ 

i>0 i>0 i>0 j>0 

These r.v.'s represent a particular realization of the decoder. The differential equations are defined for the normalized 
(i.e., divided by E) expected values of these variables. We use lower-case notation (e.g., U(t), rt(t), riij(t), etc.) for these 
deterministic trajectories. For a finite system, the decoder removes exactly one variable node in one time step of At. 

The description of peeling-style decoder is as follows. The peeling-style decoder removes one CVN or IVN in each time 
step by the following rules: 

CER: If any CN has its edges all connected to CVN's, pick one of the CVN's and remove it and all its edges. 
IER1: If any IVN has at least one edge connected to the CN's of type (0, 1), the value of the IVN is given by the attached CN 
and we remove the IVN and all its outgoing edges. 
If both CER and IER1 can be applied, then one is chosen randomly as described below. 

Since both rules remove exactly one VN, the decoding process either finishes in exactly N steps or stops early and cannot 
continue. The first case occurs only when either the IER1 or CER condition is satisfied in every time step. When the decoder 
stops early, the pattern of CVNs and IVNs remaining is known as a stopping set. We also note that the rules above, though 
described differently, are equivalent to the first node-based algorithm (LM1-NB) introduced in (TJ. 
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correctly received channel values 



CVN 



incorrectly received channel values 



IVN 





Fig. 1. Tanner graph for differential equation analysis. 



C. Analysis 

Recall that in the node-based algorithm for LM1 we have two verification rules. The first rule is that if all messages but 
one are verified at a CN, then all messages are verified. We call this type-1 incorrect-edge-removal (IER1) and this needs 
no,i (t) > to be satisfied. The second rule is: if all messages sum to zero at a CN, all messages are verified. We call this as 
correct-edge-removal (CER) in the peeling-style decoder and this requires n^o > for some i > 1. The peeling-style decoder 
performs one operation in time step. The operation is random and can be either CER or IER1. When both operations are 
possible, we choose smoothly between these two rules by picking CER with probability c\(t) and IER1 with probability C2(t), 
where 

E»>o "*.<>(*) 



ci(t) = 



T,i>o n iA t ) +"0,l(*) 
ng.ijt) 



Therefore, the differential equations can be written as 

dl 4 (t) 



dt 

dnit) 
dt 

dt 



= ci(t) 
= ci(t) 



dt 



c 2 (t) 



dt 

(i) 



c 2 (t) 



dl?\t) 

dt 
dr?\t) 



(*) 



dt 

(2) 



dt 



dn, , 

<*(*) 



(t) 



dt 



where W and ^ denote, respectively, the effects of CER and IER1. 

1) CER Analysis: If the CER operation is picked, then we choose randomly an edge attached to a CN of type (i, 0) with 
i > 1. This VN endpoint of this edge is distributed uniformly across the CVN edge sockets. Therefore, it will be attached to 
a CVN of degree k with probability ^jjy. Therefore, one has the following differential equations for and 



dt 



h(t) 
ei(t) 



(-k), for k > 1 



and 



drj'Ht) 
dt 



0. 



For the effect on check edges, we can think of removing a CVN with degree k as first randomly picking an edge of type 
(k, 0) connected to that CVN and then removing all the other k — 1 edges (called reflected edges) attached to the same CVN. 
The k — 1 reflected edges are uniformly distributed over the Ei(t) correct sockets of the CN's. Averaging over all graphs, 
the k — 1 reflected edges hit n '^j^^ CN's of type Averaging over the degree k shows that the reflected edges hit 

n i:j (t)i(a{t)-l) 



(i +J -) e , ( t) CN ' s of l yp e (^')- 

If a CN of type (i, j) is hit by a reflected edge, we lose i + j edges of type (i, j) and gain i — 1 
Hence, one has the following differential equations for j > and i + j < d c 



edges of type 



dt 



where 



(i), s _ n itj {t)i(a(t) - 1) 



{i+j)ei(t) 



One should keep in mind that nij(t) — for i + j > d c . 



s 



For n^jit) with j = 0, the effect from above must be combined with effect of the type-(i,0) initial edge that was chosen. 
So the differential equation becomes 



dt 
where 



Em>o"™,o( i ) 



2) IER1 Analysis: If the IER1 operation is picked, then we choose a random CN of type (0, 1) and follow its only edge to 
set of IVNs. This edge is attached uniformly to this set, so the differential equations for IER1 can be written as 



dr[ 2 \t) = r k (t) 

dt e r (t) 

and 

1,3 



dt °' 

(2), 

(-k), for k > 1 



where 



df 



(2) r .v _ njj(t)j(b(t)-l) 

Pi,i w 



'•' w (* + j)er(t) ' 
For riij(t) with (i, j) — (0, 1), the differential equation must also account for the initial edge and becomes 

dn$(t) 



dt 

Notice that even for (3,6) codes, there are 30 differential equations^ to solve. So we solve the differential equations numerically 
and the threshold for (3,6) code with LM1 is p* = 0.169. This coincides with the result from density evolution analysis for 
LM1-MB in [1] and hints at the equivalence between LM1-NB and LM1-MB. In the proof of Theorem 13.11 we make this 
equivalence precise by showing that the stopping sets of LM1-NB and LM1-MB are the same. 



IV. Differential Equation Analysis of LM2-NB 

We will first show the LM2-NB is equivalent to the following peeling-style decoder and then use differential equation to 
analysis the LM2-NB algorithm. The peeling-style decoder removes one CVN or IVN in each time unit by the following rules: 
CER: If any CN has its edges all connected to CVN's, pick one of the CVN's and remove it. 

IER1: If any IVN has messages from CN's with type (0, 1), the IVN and all its outgoing edges can be removed and we track 

the correct value by subtracting the value from the check node. 
IER2: If any IVN is attached to more than one CN with I-degree 1, then it will be verified and all its outgoing edges can be 

removed. 

If more than one of above are satisfied, then one is chosen randomly as described below. 

Notice that if either CER, IER1, or IER2 is satisfied in each time unit, decoding finishes in at most N time units, where 
N is the total number of variable nodes. For the same reason with LM1 case, the peeling-style decoder above is equivalent to 
the second node-based algorithm (LM2-NB) introduced in [1| (Section IV). 

The difference between LM2-NB and LM1-NB is that LM2-NB includes another verification rule: if two messages match 
at the same VN, the VN is verified and all messages connected to it are verified. In the peeling-style decoder, this can be 
interpreted as follows. If the VN is a CVN, then the case is covered in CER. If it is a IVN, this means that the IVN has more 
than 1 correct incoming message and IER2 applies. Since IVN only receives a correct message from type (i, 1) edges, this 
requires nn > for some i > 0. Since the unexposed edges in the graph are uniformly distributed over their respective VNs, 
the condition ra^i > for some i is sufficient to guarantee that a IVN satisfies IER2 with high probability. 

s There are 28 for nj j £ [0, ■ ■ ■ ,6] such that i + j < 6), 1 for r^{t), and 1 for ifc(t). 
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To smooth the choice between IER1, CER and IER2, we choose CER with probability Ci(t), IER1 with probability C2(t) 
and IER2 with probability c 3 (t) where 

Ei>o n i.oW 



ci(t) 
<*(*) 
c 3 (i) - 



Ei>O n i,oO) + Ei>O rl i,lW 
n ,l(t) 

Ei>O n i,l(*) 



Ei>O n i,o(*) + Ei>O n i,l(*) 

We use superscript ( 3 ) to denote the differential equations implied by IER2. The IER2 operation depends heavily on edges 
that connect IVNs to CNs of type (i, 1) with i > 1. For convenience, we refer to such edges as "IER2 edges". 

The main complication in the analysis of the IER2 operation is that the IER2 rule can only apply to IVNs which have 
at least two IER2 edges. Therefore, choosing such a VN uniforly skews the variable degree distribution. So, we let rj(t) = 
i^T Ei>i n i,i(*) be the fraction of IVN edges that are IER2 edges and find that is the probability that a randomly chosen 
IVN edge is an IER2 edge. For a randomly chosen IVN at time t, let the r.v. K be its degree and the r.v. L be the number of 
IER2 edges attached to it. Given that L > 2, the joint distribution of K, L is given by 

r k (t) ,k\ (nit) y (, _ „(t) \ k ~ l 
W kM =Pr(K = k,L = l\L > 2) = * U ^ " «W 

V fc r fc (t)/fcN / r,(t) \ | i *)(*) \ 

Therefore, the differential equations for l k and are 



d4 3) (*) 



dt 

and 

-v 



= o 



drf(t) -* 



= J2(-k)W kil (t), forfc> 1. 



dt , ; 

The K — L edges, which are not IER2 edges, are called reflected edges and the average number of reflected edges that hit 
a check node of degree is given by 

For n- 3 - (t) with j > 1 or = (0, 1), the differential equation is therefore 



We must also account for each of the L IER2 edges that, when removed, convert an (i, 1) CN into an (i,0) CN for some 
i > 1. For (f) with i > 0, this gives 

,( 3 )aa ^ * 



(3) 

Likewise, for n ■ with z > 0, this gives 



/V 7 fc>2 2=2 

Averaging over the CER, IER1 and IER2 operations gives the differential equations 

dt v ' dt 

dr,(t) _ drgft) „ m dr| 3) (t) 
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Solving the differential equations numerically, for the (3,6) code, gives the LM2-NB threshold p* = 0.259. This is somewhat 
larger than the threshold p* = 0.21 given by the message-based analysis in (TJ. 

V. Error Floor Analysis of LMP Algorithms 
A. The Union Bound for ML Decoding 

During the simulation of the optimized ensembles of Table [TT] we observed that there is an error floor that can be explained. 
First, we derive the union bound on the probability of error with ML decoding for the q-SC. To match our simulations with 
the union bounds, we expurgate (i.e., ignore) all codeword weights that have an expected multiplicity less than 1. 

First, we summarize a few results from [18, p. 497] that characterize the low-weight codewords of LDPC codes with degree-2 
variable nodes. When the block length n is large, all of these low-weight codewords are caused, with high probability, by short 
cycles of degree-2 nodes. For binary codes, the number of codewords with weight k is a random variable which converges 

(\ 2 p'(i)) k 

to a Poisson distribution with mean — 2fc ' . When the channel quality is high (i.e., high SNR, low error/erasure rate), the 

probability of ML decoding error is mainly caused by low-weight codewords. 

For non-binary GF(q) codes, a codeword is supported on a cycle of degree-2 nodes only if the product of the edge weights 

is 1. This occurs with probability l/{q— 1) if we choose the i.i.d. uniform random edge weights for the code. Hence, the 

number of GF(q) codewords of weight k is a random variable, denoted Bk, which converges to a Poisson distribution with 

I A2P (1) ) / n \ 
mean bk = 2 k(q-i) • After expurgating weights that have an expected multiplicity less than 1, ki — argminj ; > 1 b k > 1 

becomes the minimum codeword weight. 

The pairwise error probability (PEP) of the q-SC with error probability p is given by the following lemma. 

Lemma 5.1: Let y be the received symbol sequence assuming the all-zero codeword was transmitted. Let u be any codeword 
with exactly k non-zero symbols. Then, the probability that the ML decoder chooses u over the all-zero codeword is upper 
bounded by 

P2,k < 

Proof: See Appendix C. ■ 
Remark 5.2: Notice that bk is exponential in k and the PEP is also exponential in k. The union bound for the frame error 
rate, due to low-weight codewords, can be written as 

00 

It is easy to see fci = O(logq) and the sum is dominated by the first term bk x P2M which has the smallest exponent. When 
q is large, the PEP upper bound is on the order of O (p k )- Therefore, the order of the union bound on frame error rate with 
ML decoding is 




Pb =0 



\ip {l)p 



q\ogq 



and the expected number of symbols in error is 



O 



I 5 



B. Error Analysis for LMP Algorithms 

The error of LMP algorithm comes from two types of decoding failure. The first type of decoding failure is due to unverified 
symbols. The second one is caused by the false verification (FV). To understand the performance of LMP algorithms, we analyze 
these types of failure separately. Note that when we analyze the error caused by one reason, we do not consider the other for 
the simplicity of analysis. 

The FV's can be classified into two types. The first type is, as [1| mentions, when the error magnitudes in a single check 
sum to zero; we call this type-I FV. For single-element lists, it occurs with probability roughly 1/q (i.e., the chance that 
two uniform random symbols are equal). For multiple lists with multiple entries, we analyze the FV probability under the 
assumption that no list contains the correct value. In this case, each list is uniform on the q — 1 incorrect values. For m lists of 
size si, . . . , s m , the type-I FV probability is given by 1 — ( J 1 ^ 1 s )/ YiiLi i 9 ^ 1 )- ^ n g enera l> the Birthday paradox applies 
and the FV probability is roughly s 2 (™) / q for large q and equal size lists. 
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The second type of FV is that messages become more and more correlated as the number of iterations grows, so that an 
incorrect message may go through different paths and return to the same node. We denote this kind of FV as a type-II FV. 

Note that these are two different types of FV and one does not affect another. We cannot avoid type-II FV by increasing q 
and we cannot avoid type-I FV by constraining the number of decoding iterations to be within half of the girth (or increasing 
the girth). Fig. |2] shows an example of type-II FV. In Fig. there is an 8-cycle in the graph and we assume the variable 
node on the right has an incorrect incoming message "a". Assume that the all-zero codeword is transmitted, all the incoming 
messages at each variable node are not verified, the list size is less than s max , and each incoming message at each check node 
contains the correct message. In this case, the incorrect message will travel along the cycle and cause FV's at all variable nodes 
along the cycle. If the characteristic of the field is 2, there are a total of c/2 FV's occurring along the cycle, where c is the 
length of the cycle. This type of FV can be reduced significantly by choosing each non-zero entry in the parity-check matrix 
randomly from the non-zero elements of Galois field. In this case, a cycle causes a type-II FV only if the the product of the 
edge-weights along that cycle is 1. Therefore, we suggest choosing the non-zero entries of the parity-check matrix randomly 
to mitigate type-II FV. Recall that the idea to use non-binary elements in the parity-check matrix appears in the early works 
on the LDPC codes over GF(q) ifTTl . 

C. An Upper Bound on the Probability of Type-II FV on Cycles 

In this subsection, we analyze the probability of error caused by type-II FV. Note that type-II FV occurs only when the 
depth-2/c direct neighborhood of an edge (or a node) has cycles. But type-I FV occurs at every edge (or node). The order of 
the probability that type-I FV occurs is approximately 0(1/ q) Q]. The probability of type-II FV is hard to analyze because 
it depends on q, s max and A; in a complicated way. But an upper bound of the probability of the type-II FV is derived in this 
section. 

Since the probability of type-II FV is dominated by short cycles of degree-2 nodes, we only analyze type-II FV along cycles 
of degree-2 nodes. As we will soon see, the probability of type-II FV is exponential in the length of the cycle. So, the error 
caused by type-II FV on cycles is dominated by short cycles. We also assume s max to be large enough such that an incorrectly 
received value can pass around a cycle without being truncated. This assumption makes our analysis an upper bound. Another 
condition required for an incorrectly received value to participate in a type-II FV is that the product of the edge weights along 
the cycle is 1. If we assume that almost all edges not on the cycle are verified, then once any edge on the cycle is verified, 
all edges will be verified in the next k iterations. So we also assume that nodes along a cycle are either all verified or all 
unverified. 

We note that there are three possible patterns of verification on a cycle, depending on the received values. The first case 
is that all the nodes are received incorrectly. As mentioned above, the incorrect value passes around the cycle without being 
truncated, comes back the node again and falsely verifies the outgoing messages of the node. So all messages will be falsely 
verified if they are all received incorrectly after k iterations. Note that this happens with probability -^zjp k - The second case is 
that all messages are verified correctly, say, no false verification. Note that this does not require all the nodes to have correctly 
received values. For example, if any pair of adjacent nodes are received correctly, it is easy to see all messages will be correctly 
verified. The last case is, there is at least 1 incorrectly received node in any pair of adjacent nodes and there is at least 1 
node with correctly received value on the cycle. In this case, all messages will be verified after k iterations, i.e., messages 
from correct nodes are verified correctly and those from incorrect nodes are falsely verified. Then the verified messages will 
propagate and half of the messages will be verified correctly and the other half will be falsely verified. Note that this happens 
with probability ^rj2 {j> k l 2 — p k ) ~ 2 q-i anc ^ tms approximation gives an upper bound even if we combine the previous 
—^-rP k term. 

(A2P / (1)) 

Recall that the number of cycles with length k converges to a Poisson with mean 2fc ■ Using the union bound, we 
can upper bound on the ensemble average probability of any type-II FV event with 



Pr(any type-II FV) < £ \ - y 2p 



2*(,-l)-" £ 



The ensemble average number of nodes involved in type-II FV events is given by 

~ (a 2 p'(i)) „ ~ (x 2 p(l)^p) 

E [symbols in type-II FV] < > '—2kp^ = > ^ '- 



, log q y 



( (\ 2 p'(l)^p) ° E . 

The upper bound on the frame error rate of type-II FV is on the order of O I ^ — ( q -i) i og g — I ™ tne u PP er bound on the 

/ (a2p'(i)Vp) \ 

ensemble average number of nodes in type-II FV symbol is on the order of O I - — ^ — - I . Notice that both bounds are 
decreasing functions of q. 
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Fig. 2. An example of type-II FV's. 




-a 



D. An Upper Bound on the Probability of Unverification on Cycles 

In this subsection, We derive the union bound for the probability of decoder failure caused by the symbols on short cycles 
which never become verified. We call this event as unverification. As described above, we can always pick q large enough to 
have arbitrarily small probability of both type-I and type-II FV. In this case, the error is dominated by the unverified messages 
because the following analysis shows that the union bound on the probability of unverification is independent of q. 

In contrast to type-II FV, unverification event does not require cycles, i.e., unverification occurs even on subgraphs without 
cycles. But in the low error-rate regime, the dominant unverification events occur on short cycles of degree-2 nodes. Therefore, 
we only analyze the probability of unverification caused by short cycles of degree-2 nodes. 

Consider a degree-2 cycle of length k and assume that no FV occurs in the neighborhood of this cycle. Assuming the 
maximum list size is s max , the condition, which is denoted as UV, for unverification is that there is at most one correctly 
received value along s max + 1 adjacent variable nodes. Note that we don't consider type-II FV since type-II FV occurs with 
probability and we can choose q to be arbitrarily large. On the other hand, unverification does not require the product 
of the edge weights on a cycle to be 1, so we cannot mitigate it by increasing q. So the union bound on the probability of 
unverification on a cycle with length k is 

~ (V(i)) fe 

^ 2k 

k>k 2 



Pu < 



"^(•SmouP) k) 



where k 2 — argmirife>i 



(W(i))' 

2k 



> 1 and 6{s r 



, p, k) is the UV probability which is given by the following lemma. 



Lemma 5.3: Let the cycle have length k, the maximum list size be s, and the channel error probability be p. Then, the 
probability of an unverification event on a degree-2 cycle of length-fc is (f>(s,p 7 k) = Tr (B k (p)^ where B{p) is the (s + 1) by 
(s + 1) matrix 

" P 1 ~P 



B(p) 








p 





p 










p 





(18) 



Proof: See Appendix D. 
Finally, the union bound on the average number of symbols involved in unverification events is 



~ (A 2 //(l)) fc 
E [unverified symbols] < ^ 4>(s max ,P,k). 



(19) 



k>k 2 



VI. Comparison and Optimization 

In this section, we compare the proposed algorithm with maximum list size S (LMP-S*) with other message-passing decoding 
algorithms for the q-SC. We note that the LM2-MB algorithm is identical to SW1 for any code ensemble because the decoding 

TABLE II 

Optimization Results for LMP Algorithms (rate 1/2) 



Alg. 


A(x) 


pW 


P* 


LMP-1 


. 1200x + .3500x z + .0400x 4 + .4900x 14 


x s 


.2591 


LMP-1 


. 1650x + .3U5x' A + .0085x 4 + .2111x i4 + .0265x^ 4 + .OOTO^ 4 + .2674z 4y 


.0030^ + .9970x iu 


.2593 


LMP-8 


.32i: + .24^ + .262: s + .19a; 14 


.02x 4 + .82x D + .162: s 


.288 


LMP-32 


A0x+.20x i + .13x b + .04:X> i + .23x'-' 1 


.04x 4 + .96x b 


.303 


LMP-oo 


.34x + .16^ + .2l2: 4 + .29a; 14 


x> 


.480 


LM2-MB 


.2x + .3x- > + .05x t > + A5x 11 




.289 
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TABLE III 

Threshold vs. algorithm for the (3,6) regular LDPC ensemble 



LMP-1 


LMP-8 


LMP-32 


LMP-oo 


LMl 


LM2-MB 


LM2-NB 


.210 


.217 


.232 


.429 


.169 


.210 


.259 



rules are the same. LM2-MB, SW1 and LMP-1 are identical for (3,6) regular LDPC codes because the list size is always 1 
and erasure never happens in LMP-1 for (3,6) regular LDPC codes, and the LMP-oo algorithm is identical to SW2. 

There are two important differences between the LMP algorithm and previous algorithms: (i) erasures and (ii) FV recovery. 
The LMP algorithm passes erasures because, with a limited list size, it is better to pass an erasure than to keep unlikely 
symbols on the list. The LMP algorithm also detects FV events and passes an erasure if they cause disagreement between 
verified symbols later in decoding, and can sometimes recover from a FV event. LM1-NB and LM2-NB fix the status of a 
variable node once it is verified and pass the verified value in all following iterations. 

The results in HI and |[T4l also do not consider the effects of type-II FV. These FV events degrade the performance in 
practical systems with moderate block lengths, and therefore we use random entries in the parity-check matrix to mitigate 
these effects. 

Using the DE analysis of the LMP-S* algorithm, we can improve the threshold by optimizing the degree distribution pair 
(A, p). Since the DE recursion is not one-dimensional, we use differential evolution to optimize the code ensembles |fl9l . In 
Table ITR we show the results of optimizing rate-i ensembles for LMP with a maximum list size of 1, 8, 32, and oo. Thresholds 
for LMl and LM2-NB/MB with rate 1/2 are also shown. In all but one case, the maximum variable-node degree is 15 and the 
maximum check-node degree is 9. From Table ITR one can see that the resulting code ensembles have concentrated check-node 
degrees. The second table entry allowed for larger degrees (in order to improve performance) but very little gain was observed. 
We can also see that there is a gain of between 0.05 and 0.07 over the thresholds of (3,6) regular ensemble with the same 
decoder. 

VII. Simulation Results 

In this part, we show the simulation results for (3,6) regular LDPC codes using various decoding algorithms as well as 
the simulation results for the optimized ensembles shown in Table II with LMP algorithms in Fig. IVIIII In the simulation of 
optimized ensembles, we try different maximum list sizes and different finite fields. We use notation "LMPs,<7,ensemble" to 
denote the simulation result of LMP algorithm with maximum list size s, finite field GF(q) and the simulated ensemble. We 
choose block length to be 100000. The parity-check matrices are chosen randomly without 4-cycles. Each non-zero entry in the 
parity-check matrix is chosen uniformly from GF(g) \ 0. This allows us to keep the FV probability low. The maximum number 
of decoding iterations is fixed to be 200 and more than 1000 blocks are run for each point. These results are compared with 
the theoretical thresholds. Table ITTT1 shows the theoretical thresholds of (3, 6) regular codes on the q-SC for different algorithms 
and Table II shows the thresholds for the optimized ensembles. The numerical results match the theoretical thresholds very 
well. 

In the results of (3,6) regular codes simulation, we cannot see any error floor because there is almost no FV in the simulation. 
The LM2-NB performs much better than other algorithms with list size 1 for (3,6) regular ensemble. But in the results of 
the optimized ensembles, the error floors occur because the number of degree-2 variable nodes the maximum variable/check 
degrees are significantly larger than the (3,6) regular ensemble. By evaluating Eq. ( fT9l , the predicted error floor caused by 
unverification is 1.6 x 10~ 5 for the optimized s max — 1 ensemble, 8.3 x 10~ 7 for the optimized s ma x — 8 ensemble, and 
1.5 x 10 -6 for the optimized s max = 32 ensemble. From the results, we see the analysis of unverification events matches the 
numerical results very well. 

VIII. Conclusions 

In this paper, we discuss list-message-passing (LMP) decoding algorithms for the g-ary symmetric channel {q-SC). It is 
shown that capacity-achieving ensembles for the BEC achieve capacity on the q-SC when the list size is unbounded. Decoding 
thresholds are also calculated by density evolution (DE). We also derive a new analysis for the node-based algorithms described 
in (TJ. The causes of false verification (FV) are analyzed and random entries in the parity-check matrix are used to mitigate 
avoid type-II FV. Degree profiles are optimized for the LMP decoder and reasonable gains are obtained. Finally, simulations 
show that, with list size larger than 8, the proposed LMP algorithm outperforms previously proposed algorithms. 

While we focus on the q-SC in this work, there are a number of other applications of LMP decoding that are also quite 
interesting. For example, the iterative decoding algorithm described in [20] for compressed sensing is actually the natural 
extension of LMl to continuous alphabets. For this reason, the LMP decoder may also be used to improve the threshold of 
compressed sensing. This is, in some sense, more valuable because there are a number of good coding schemes for the q-SC, 
but few low-complexity near-optimal decoders for compressed sensing. 
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Fig. 3. Simulation results for (3,6) regular codes with block length 100000. 

Appendix A 
Proof of theorem I2.1I 

Proof: Given p\(l — p(l — x)) < x for x £ (0, 1], we start by showing that both ir, and go to zero as i goes to infinity. 
To do this, we let a = sup^Q ^ ^pA(l — p(l — x)) and note that a < 1 because p < p*. It is also easy to see that, starting 
from xq — 1, we have xi < a 1 and Xi — > 0. Next, we rewrite Eq. (01 as 

Vi+i = -Xi+i - Xi) - p(l - yij) A'(l - p(l - x^) 

P 

< +P (1 - p'(1K - P(l - Vi)) (A 2 + 0(« 1 )) 

(b) 1 

< -a^+pACi-pa-y^Ci + o^)) 

< ~a t+1 + a Vi (1 + 0(a 1 )) , 

where (a) follows from p(l — < 1 — p'(l)x, (b) follows from A2(l — p(l — y)) < A(l — p(l — y)), and (c) follows from 
pA(l - p(l - y)) < ay. It is easy to verify that y i+1 < yi as long as y l > p(1 _ a ^ +0(ai)) y Therefore, we find that y { 
because the recursion does not have any positive fixed points as i — > oo. Moreover, one can show that yi eventually decreases 
exponentially at a rate arbitrarily close to a. 

Note that the decoding error comes from two reasons, one is the event that message is not verified and the other one is 
the event that the message is falsely verified. Next, we are going to show that the actual performance of a code converges to 
the ensemble average exponentially which means almost every code in a capacity-achieving ensemble has capacity-achieving 
performance. Note that the concentration effect and the decay of FV probability hold regardless the error probability of the 
decoder converges to zero or not. 

We can prove that the performance of a particular code converges to the threshold which is the average performance of a 
tree-like ensemble in a similar way in [15|, where the average is over the graph ensemble (X(x), p(x)) and all the channel 
inputs. There are two difference between our scenario and ||T5l . i.e, our algorithm passes a list of values with unbounded list 
size, the second difference is the graph may be irregular in our case. Here we only mention the brief procedure of the proof. 
We can let Z^/E denote the fraction of unverified messages at the l-th iteration, where E is the number of edges in the 
graph. Note that denotes the number of incorrect and erasure messages in lfT31 . Following [15], we can break failure the 
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probability into a tree-like neighborhood term and a Martingale concentration term to get 



Pr 



E 



Vl 



>e < 



Pr 



gO(s) E[Z(')(s)] 



> e/2 + 



Pr 



E[ZW(b)] 



J// 



> e/2 



where s is an arbitrary codeword chosen from ensemble (A, p), Z'''(s) is the random variable that denotes the number of 
erroneous variable-to-check messages after I decoding iterations. E is the number of edges in the graph. This means the 
concentration theorem can be think of consisting of two parts of concentration, i.e., concentration from a particular code to 
the ensemble with cycles and concentration from a ensemble with cycles to the tree-like ensemble. Notice that the proof of 
the later concentration and the proof of the probability of a tree-like neighborhood are not limited to the specific decoding 
algorithm and the definition of Z^>, the proof is omitted here. By forming a Doob's martingale on the edge-exposure and 
applying Azuma's inequality, we can prove the concentration from a particular code to the ensemble in the same manner as 
lfT31 . In our scenario, the proof of bounded difference of the martingale, the right hand side of in [15 Eq. (16)] should be 



the cardinality of depth 21 directed neighbor of e, 



Ki 2 



The right hand side of in lfl5l Eq. (17)] should be 4 



a/; 
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ft in applying Azuma's inequality is X)fLi \A Ne 2 ^ ^ + Sfe=i \A 
constant e/2, there exist positive numbers j3 and 7, such that if n > then 



(21) 



The 



So far, we prove that, for an arbitrary small 



Pr 



E 



m 



> e < e 



Note that the similar proof can be found in [15| (the proof of Theorem 2) and ||2T1 (the proof of Theorem 1). Note that 031 
proves for the regular code ensemble and ED extends the proof to the irregular code ensemble. So, for an arbitrary code s 
and an arbitrary small quantity e, the fraction of unverified message is less than e/2 as n goes to infinity. 

In lfl5l and lED . it is proved that, when a code graph is chosen uniformly at random from all possible graphs with degree 
distribution pair (X(x), p(x)), 

Pr (neighborhood of depth 21 is not tree-like) < — 

n 

where 7 is a constant independent of n. So, given e, we can choose n large enough such that the number of variable nodes 
which are involved in cycles of length less than 21 is less than ne/2 with probability arbitrarily close to one as n goes to 
infinity. So the probability of error caused by type-II FV's is upper bounded by e/2 (for the notation of type-I and type-II FV, 
please refer to Section |V-Bt . Here, we don't consider the type-I FV's because the probability of type-I FV's can be forced 
arbitrarily close to zero by choosing a large enough q. ■ 



Appendix B 
Proof of Theorem I3.1I 

The verification occurs in LM1-NB also occurs in LM1-MB and vice versa. So LM1-NB and LM1-MB are equivalent. 
Proof: The operations of LM1-MB and LM1-NB are different because they have different verification rules (see Table. I). 
We can prove they are equivalent by showing the verification occurs in LM1-MB also occurs in LM1-NB and vice versa, but 
in different decoding steps. Let's first look at the check node when the summation of all messages equals to zero but there are 
more than 1 messages are unverified. In this case, LM1-NB will verify all the messages. In LM1-MB none of them will be 
verified but all the values will be correct. In the following iteration, all these messages will be verified on their variable nodes. 
Notice that this is the only case verification occurs in LM1-NB but not in LM1-MB. So verification in LM1-NB also occurs 
in LM1-MB. Let's then look at the variable node when any incoming message is correct and the channel value is correct. In 
LM1-MB, the outgoing message will be verified. In LM1-NB the message will be correct but not verified. Notice that the 
incoming is correct means all the other messages are correct at the check node, so the unverified correct message will be 
verified in the next step on check node. Notice that this is the only case verification occurs in LM1-MB but not in LM1-NB. 
So verification in LM1-MB also occurs in LM1-NB. ■ 
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Fig. 4. Finite-state machine for Lemma 1531 

Appendix C 
Proof of Lemma [5TT1 

Let y be the received symbol sequence assuming the all-zero codeword was transmitted. Let u be any codeword with exactly 
k non-zero symbols. It is easy to verify that the probability that ML decoder chooses u over the all-zero codeword is given by 

-gSU-.->-«'(^)'(^^ 

Using the multinomial theorem, it is also easy to verify that 



k 



k k-j 



q-l q-l 

k Vi si( P V fp(q-2)\ k - 1 ^ k 



EEL <)v-p) 

j=0 i=0 



i,3,k-i-jj \q-lj \ q-l 



2k 

A 



E^. 



1=0 

where Ai is the coefficient of x l in A(x). Finally, we observe that p2.fc = Sf=fe Ai is simply an unweighted sum of a subset 
of terms in A(x) (namely, those where k — i + j > k). 
This implies that 

2k 

X k p2.k =Yl Alxk ~ A ^ 
l=k 

for any x > 1 . Therefore, we can compute the Chernoff-type bound 

P2,k < inf x~ k A(x). 

X>1 

By taking derivative of x~ k A(x) over x and setting it to zero, we arrive at the bound 

fe 




Appendix D 
Proof of Lemma [531 

Proof: An unverification event occurs on a degree-2 cycle of length-A: when there is at most one correct variable node in 
any adjacent set of s + 1 nodes. Let the set of all error patterns (i.e., means correct and 1 means error) of length-fc which 
satisfy the UV condition be $(s,p, k) C {0, l} fe . Using the Hamming weight w(z), of an error pattern as z, to count the 
number of errors, we can write the probability of UV as 

<Ka,p,k)= P w{z) 0--P) k - W{z) - 

This expression can be evaluated using the transfer matrix method to enumerate all weighted walks through a particular 
digraph. If we walk through the nodes along the cycle by picking an arbitrary node as the starting node, the UV constraint 
can be seen as fc-steps of a particular finite-state machine. Since we are walking on a cycle, the initial state must equal to the 
final state. 

The finite-state machine, which is shown in Fig. |4] has s + 1 states {0, 1, . . . , s}. Let state be the state where we are 
free to choose either a correct or incorrect symbol (i.e., the previous s symbols are all incorrect). This state has a self-loop 
associated with the next symbol also being incorrect. Let state i > be the state where the past i values consist of one correct 
symbol followed by i — 1 incorrect symbols. Notice that only state may generate correct symbols. By defining the transfer 
matrix with ( fT8l l. the probability that the UV condition holds is therefore 4>(s,p, k) = Tr (_B fc (p)). ■ 
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